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SECTION  I 


INTRODUCTION 

Although  the  Air  Force  accident  rate  in  recent  years  has  been  at  an 
all  tine  low,  many  feel  that  an  even  lower  rate  could  be  achieved  by 
reducing  the  number  of  controlled  flight  into  terrain  (CFIT)  accidents, 
which  constitute  the  second  largest  category  of  tactical  Air  Force  (TAF) 
Class  A  mishaps.  Air  Force  investigations  concluded  that  between  1975 
and  1981,  there  were  56  fighter  and  attack  aircraft  involved  in  CFIT 
accident  mishaps.  The  United  States  Navy  has  attributed  the  loss  of  317 
aircraft  to  CFIT  accidents  between  1970  and  1984.  Of  course,  the  occur¬ 
rence  of  CFIT  could  be  greatly  reduced  in  the  tactical  fleet  by  in¬ 
stalling  forward-looking  Terrain-Following  Radar  (TFR)  systems,  but  such 
a  solution  would  be  prohibitively  expensive  and  possibly  unnecessary. 
Since  a  significant  percentage  of  accidents  occur  during  low  level 
missions  while  flying  over  sloping  terrain  that  is  not  particularly 
severe,  it  has  been  proposed  that  a  less  costly  solution  to  the  problem 
may  be  a  system  based  on  less  complicated  sensors,  such  as  a  radar 
altimeter  system,  that  would  cover  a  part  of  the  CFIT  envelope  instead  of 
all  of  it. 

Such  an  approach  has  been  highly  successful  on  commercial  airliners 
and  some  wide  body  military  transports  which  are  equipped  with  a  Ground 
Proximity  Warning  System  (GPWS).  The  CFIT  accidents  for  commercial 
aircraft  have  in  fact  dropped  to  virtually  zero  since  1976,  when  GPWS  use 
was  mandated  by  the  Federal  Aviation  Administration.  However,  the  much 
more  complex  nature  of  tactical  application  cannot  be  accomodated  by  the 
system  currently  available  in  conmercial  and  wide-body  aircraft.  In  an 
effort  to  develop  a  Ground  Collision  Avoidance  System  (GCAS)  which  may 
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have  application  to  the  ouch  more  complex  tactical  mission,  the  Air  Force 
haa  contracted  with  Cubic  Corporation  to  develop  a  generic  GCAS  system 
that  is  software  oriented  with  a  minimum  of  complex  sensors  required. 

Such  a  system  was  developed  and  has  been  flight  tested  on  a  limited 
basis.  Due  to  safety  considerations  and  in  order  to  assess  GCAS  in  a 
wider  variety  of  tactical  situations,  the  CSDF  was  requested  to  conduct  a 
simulation  study  of  the  generic  GCAS.  The  overall  objective  of  the 
simulation  study  was  to  evaluate  GCAS  in  a  variety  of  tactical  scenarios. 
More  specifically,  the  purpose  of  the  simulation  was  to  provide  the 
following  data  to  the  GCAS  program  office: 

a.  Evaluate  the  adequacy  of  the  algorithm  to  provide  a  warning 
in  sufficient  time  for  a  pilot  to  recover  in  a  variety  of 
CFIT  Scenarios. 

b.  Evaluate  the  impact  and  incidence  of  nuisance  warnings  gen¬ 
erated  by  the  algorithm. 

c.  Provide  data  to  further  refine  and  flight  test  the  algorithm. 

d.  Assets  pilot  reaction  to  GCAS. 

e.  Provide  normative  data  on  pilot  response  time  to  GCAS 
warnings. 
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SECTION  II 


STUDY  PROCEDURE 

1.  APPARATUS 

The  apparatus  for  conducting  this  experiment  was  the  Crew  Station 
Design  Facility,  which  has  the  capability  to  dynamically  simulate  a 
complete  flight  regime  under  a  variaty  of  controlled  conditions.  The 
facility  consists  of  six  basic  components: 

a.  Three  crew  station  shells  -  A-10,  F-16,  and  C-135/C-18. 

b.  A  digital  computer  complex  consisting  of  five  Gould  Model 
77/80SEL  computers,  one  Gould  Model  8780  SEL  computer  «nd  i 
Digital  Equipment  Corporation  Model  PDP  11/34.  All  cockpits 
are  interfaced  through  a  Singer  Advanced  Simulator  Technology 
(AST)  interface. 

c.  A  visual  simulation  complex  consisting  of  a  Singer  Link  NVS 
Night  Caligraphic  Visual  System  and  Singer  Dual  SMK-23  Cmnera 
Model  Visual  System. 

d.  Radar  simulator  equipment. 

e.  Monitors  and  Recording  equipment. 

f.  Computer  graphics  complex. 

The  CSDF  F-16  Flight  Simulator  was  used  as  the  test  vehicle  for 
this  study.  An  out-of-the-cockpit  visual  scene  was  provided  using  a 
closed-circuit  TV  system  from  a  modified  Link  SMK-23  moving  terrain  model. 
The  visual  system  consisted  of  a  high  resolution,  low  light  level  TV 
camera  and  a  Farrand  optical  probe,  which  transferred  the  mountainous 
terrain  images  to  a  Conrac  1000  line,  black  and  white  TV  monitor.  The 
scene  was  transmitted  through  a  beam  splitter  and  parabolic  mirror  with  a 


3 


focal  length  of  54  inchaa.  Thia  provided  approximately  a  48-degrse-for- 
vard  field  of  view  (FOV)  to  the  pilot  with  the  image  collimated  to  appear 
at  infinity.  The  ayatam  provided  simulated  aircraft  viaual  parametara  of 
360  degreea  continuoua  heading,  360  dagreea  of  continuoua  roll,  plus  or 
minua  120  degreea  of  pitch,  and  50  to  4000  feet  altitude.  All  aircraft 
aerodynamica  and  aircraft  system  were  computed  on  the  SII.  coaf>uter  com¬ 
plex.  The  Voice  Warning  Syatem  wan  uaad  to  tranamit  digitised  warnings 
(pull-up,  pull-up)  to  the  pilot  for  each  GCAS  event.  Objective  data 
parameters  collected  on  each  mission  were  taken  from  the  S5L  computers 
and  recorded  on  magnetic  tape  for  subsequent  review  and  analysis. 

Figures  1  and  2  provide  an  illustration  and  diagram  of  the  apparatus. 

The  experimenter 's  console  provides  controls  and  displays  for  mission 
setup,  simulator  operation,  data  collection  equipment,  and  various 
control  and  display  mechanisation  options.  For  this  evaluation  there 
were  three  options  available  to  the  experimenter  to  change  simulator 
position  and  attitude.  One  option  allowed  the  console  operator  to  move 
the  simulator  forward  3000  feet  in  the  flight  plan  to  simulate  a  late 
pop-up  on  weapon  delivery  runs.  When  this  occurs  in  the  real  world,  the 
pilot  has  no  difficulty  finding  his  pull-up  point,  pulls  up  too  close  to 
the  target  and  ends  up  making  his  dive-bomb  pass  at  a  steeper  than  normal 
delivery  angle.  This  feature  was  used  on  missions  14  and  15  (auto  pull-up 
missions  24  and  25)  to  complicate  the  delivery  task. 

Another  option  allowed  him  to  decrease  simulator  altitude  in 
increments  of  1000  feet.  This  was  used  most  often  on  mission  16  (auto 
26)  to  simulate  entry  into  dynamic  maneuvers  at  a  lower  than  expected 
altitude.  This  led  to  unexpected  GCAS  warnings  in  attitude  and  flight 
path  conditions  approaching  the  limits  of  system  operation  and  in  some 
cases,  disorientation. 
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The  final  option  dealing  with  simulator  control  was  the  mechani¬ 
sation  of  a  joy  stick  on  the  console  to  operate  in  series  with  the 
pilot's  control  stick.  With  this  capability,  the  experimenter  could 
change  aircraft  pitch  and  roll  attitudes  during  periods  of  pilot  distrac¬ 
tion  to  simulate  entry  into  unusual  (or  unexpected)  attitudes.  This 
control  feature  was  used  effectively  during  low  level  navigation  and  high 
"G"  turning  flight  when  pilots  were  instructed  to  read  authentication  and 
alphanumeric  cards. 

2.  SUBJECTS 

A  total  of  10  subject  pilots  participated  in  the  study.  Eight  of 
the  pilots  were  operational  pilots  from  the  Tactical  Air  Command  (TAC), 
and  two  were  pilots  from  Wright -Patterson  AFB  with  previous  tactical 
experience  in  the  low  level  environment.  The  pilots  had  varying  amounts 
of  experience  in  the  F-16  as  indicated  in  Table  1.  They  arrived  over  a 
2-1/2  week  period  with  an  average  of  four  pilots  participating  each 
week  (two  subjects  over  a  2-1/2  day  period). 

3 .  PROCEDURE 

When  the  pilots  arrived  to  participate  in  the  study,  they  filled  out 
a  personal  data  questionnaire  (see  Appendix)  and  were  given  a  general 
overall  briefing  on  the  program.  A  detailed  briefing  on  the  GCAS  algo¬ 
rithm  was  also  presented  along  with  "ground  school"  on  the  F-16  sim¬ 
ulator.  The  pilots  flew  a  training  mission  in  addition  to  the  briefings 
to  conclude  the  inital  half-day  period.  In  the  er  uing  day  and  a  half, 
the  pilots  flew  12  missions  in  which  various  data  described  earlier  were 
collected.  In  the  last  half-day  block,  each  pilot  was  debriefed  and 
filled  out  a  questionnaire.  A  detailed  schedule  of  events  for  each  pair 
of  pilots  is  in  the  Appendix. 
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TAILS  1 


PILOT *8  FLYING  TIME 

8UBJ2CT 

NUMBER 

TOTAL 

FLYING  TIME  (HRS) 

TOTAL 
F-16  TIME 

1 

350 

150 

2 

400 

110 

3 

1500 

1250 

4 

1700 

350 

5 

600 

350 

6 

300 

200 

7 

1780 

780 

8 

1700 

640 

9 

2000 

600 

10 

1170 

0 

M-  1155 

443 

Pilot  briefings  prior  to  each  data  collection  flight  covered  the 
overall  mission  scenario,  specific  objectives  of  the  mission,  altitudes, 
speeds,  times,  etc.  Pilots  were  also  provided  with  a  mission  setup 
checklist  and  a  strip  map  of  the  route  of  flight  when  appropriate. 
Mission  briefing  outlines  are  provided  in  the  Appendix.  Following  the 
briefing,  each  pilot  was  taken  to  the  simulator  to  fly  the  test  mission. 

When  the  pilot  had  completed  cockpit  setup  actions,  previewed 
the  mission  and  indicated  that  he  was  ready  to  start  the  run,  the 
simulator  and  data  recording  equipment  were  activated  and  the  mission 
started.  Voice  conmunicationa  were  held  to  a  minimum  during  each 
evaluation  run  and  limited  to  responding  to  the  pilots'  questions,  or  as 
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was  the  case  with  missions  11  and  15  (auto  21  and  25 ),  recording  hia 
assessment  of  the  validity  of  GCAS  warnings,  since  these  were  the  only 
missions  where  he  was  in  a  position  to  assess  validity  of  the  warning  as 
valid,  invalid  (expected),  or  invalid  (unexpected). 

Following  each  simulated  mission,  a  short  debriefing  session 
provided  an  opportunity  for  the  experimenter  and  evaluation  pilot  to 
discuss  simulator  and  GCAS  operation.  During  these  sessions,  the  experi¬ 
menter  recorded  pilot  comments  on  GCAS  operation  as  it  applied  to  the 
mission  just  flown.  The  questionnaire  with  pilot  comments  can  be  found 
in  the  Appendix. 

4.  DATA  COLLECTION 

Table  2  shows  the  flight  parameters  and  performance  measures  that 
were  collected  during  all  test  missions.  These  parameters  were  to  recon¬ 
struct  the  flights  for  more  definitive  evaluation.  The  critical  flight 
parameters  were  plotted  as  time  histories  for  each  maneuver  causing  a 
warning.  The  data  were  plotted  from  a  period  of  5  seconds  prior  to 
the  warning  to  the  minimum  recovery  altitude  above  ground.  These  data 
plots  were  used  to  reconstruct  each  GCAS  event  for  data  analysis  and 
assessment  of  pilot  performance. 

5.  EXPERIMENTAL  DESIGN 

The  experimental  design  was  a  treatment  x  treatment  x  subjects 
(2x2x10)  or  ABS  design  as  described  in  Lindquist's  Design  and  Analysis  of 
Experiments  in  Psychology  and  Education  (Ref  1).  The  treatment 
variables  were  (1)  type  of  recovery  (manual  or  automatic)  and  (2)  type  of 
mission  (six  representative  tactical  missions).  The  manual  pull-up  data 
were  used  to  assess  pilot  performance  and  the  automatic  pull-up  data  were 
to  analyze  the  algorithm  (i.e.,  provide  consistent  response  without 
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TABLE  2  FLIGHT  PARAMETERS 


FLIGHT  PARAMETERS  UNIT  OF  MEASUREMENT  RANGE 

Pitch  Attitude  degrees  +90 

Roll  Attitude  degrees  +180 

"G"  load  "G"  -3“tc  +10 


Flight  Path  Angle 
Pitch  Rate 
Absolute  Altitude 
Barometric  Altitude 

Heading 
Roll  Rate 
Yaw 

Terrain  Angle 

GCAS  triggered  Pull-up  time 

Altitude  at  GCAS  alara 

Altitude  clearance  at 
pull-up  instance 

Engine  RPM 

Groundspeed 

True  Airspeed 

Angle  of  Attack 

Stick  Force 

Time  from  alarm  to  change  in 
stick  force  (response  time) 


degrees 

_+90 

degrees/sec 

±45 

feet 

0-1000 

feet 

0-6000 

degrees 

0-360 

degrees /sec 

+75 

degrees 

+45 

degrees 

+60 

sec 

Discrete 

feet 

Discrete 

feet 

Discrete 

ZRPM 

Idle-95 

knots 

0-600 

knots 

0-600 

degrees 

lbs 

Discrete 

clock 

Seconds 
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variations  in  response  time) 


6.  MISSIONS 

Six  missions  were  designed  to  test  the  algorithm.  Overall 
objectives  in  designing  the  profiles  were  to  subject  the  pilot  to  a 
variety  of  conditions  representative  of  those  encountered  in  tactical 
fighter  operations  with  specific  emphasis  on  the  kinds  of  situations 
reported  in  CFIT  incidents.  A  complete  description  of  each  mission 
follows: 

1.  Low  Level  Navigation  -  Mission  11:  This  mission  was  flown  at 
300  to  500  feet  above  the  terrain.  Some  segments  of  the  route  were 
level  terrain,  between  ridges,  simulating  typical  terrain  masking  oper¬ 
ations  while  other  segments  required  ridge  crossings  and  flight  over 
rough  terrain  (Figures  3  and  4). 

2.  Low  Speed  Maneuvering  IMG  -  Mission  12:  This  mission  was  designed 
to  simulate  radar  vectors  for  approach  or  low  level,  low  speed, 
navigation  where  the  controller  or  the  pilot  selected  a  maneuvering 
altitude  below  minimum  safe  altitude  and  slightly  below  the  level  of 
ridges  along  the  route  of  flight.  Flown  at  1500  feet  MSL  (altitude  below 
ridgetops),  250KCAS  and  in  instrument  meteorlogical  conditions,  this 
mission  looked  at  the  algorithm  in  a  nearly  level  flight,  wings  level 
condition  where  the  pilot  had  no  external  visual  cues  from  outside 
terrain. 

3.  Hard  Turns  at  Low  Altitude  -  Mission  13:  This  mission  was  flown 
over  level  terrain  at  480KCAS  in  a  figure-eight  pattern  between  two 
steerpoints.  At  each  steerpoint  passage,  the  pilot  initiated  a  5  ' G '  turn 
of  about  210  degrees  to  simulate  defensive  maneuvering  at  low  level 

300  to  500  feet).  During  the  turns,  pilots  were  asked  to  read  a  series 
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of  numbers  and  letters  from  a  card  located  on  the  canopy  over  his  head  to 
simulate  checking  6  o'clock  for  other  aircraft.  Adjustments  to  aircraft 
attitude  were  made  by  the  experimenter  during  the  turns  on  this  mission  to 
further  complicate  the  recovery  task  (Figure  5). 

4.  Range  Mission  -  Mission  14:  This  mission  consisted  of  a  series 
of  three  runs  on  the  same  target  from  different  approach  headings  simu¬ 
lating  a  low  level  range  familiarisation  flight.  The  profiles  were  flown 
at  300  to  500  feet  and  480  KCAS  on  the  low  level  navigation  segments. 
Delivery  maneuvers  consisted  of  a  30-degree  straight  ahead  pop  at  20  sec¬ 
onds  prior  to  the  target,  a  climb  to  4000  feet  MSL  and  a  roll  to  inverted 
flight  for  the  pull  down  to  a  30-degree  dive.  Bomb  release  was  briefed  to 
be  completed  at  2500  feet  MSL.  To  cooplicate  the  delivery,  the  experi¬ 
menter  could  move  the  aircraft  (simulation)  forward  3000  feet  during  the 

a  steeper  then  desired,  or  expected,  dive  angle  that  had  to  be  used  to 
bring  the  target  into  view  (Figures  6  and  7). 

5.  Lew  Level  Navigation/Pop-Up  Deliveries  -  Mission  15:  The  low 
level  portions  of  this  mission  were  essentially  the  same  as  described  in 
Mission  11  above  except  that  three  targets  were  selected  along  the  route 
of  flight  for  pop-up  weapons  delivery  maneuvers.  Like  the  range 
mission  above,  the  pop  was  initiated  20  seconds  prior  to  the  target  for  a 
30  degree  delivery.  Again,  the  experimenter  could  advance  the  simulator 
in  the  pop  so  that  the  aircraft  would  be  outside  delivery  parameters  in 
the  pull-down  (Figures  8  and  9). 

6.  IMC  Maneuvering  -  Mission  16:  This  mission  flown  in  IMC  was  made 
up  of  a  series  of  aerobatic  maneuvers.  These  maneuvers  required  the 
pilot  to  recover  from  relatively  extreme  pitch  and  bank  attitudes  such  as 
might  be  encountered  in  unusual  attitude  or  spatial  disorientation  inci¬ 
dents  (Figure  10). 
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On  so me  of  the  manuevers  such  as  the  loop,  split  s,  and  Cuban  8,  the 
experimenter  adjusted  simulator  altitude  3000-4000  feet  to  cause  early 
GCAS  alarms. 
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Mission  14  (24)  Range  Mission  (Artist’s  Concept) 


SECTION  III 


STUDY  RESULTS 

1.  PERFORMANCE  DATA 

Table  3  shows  Che  mean  pilot  response  time  for  making  a  control 
input  following  a  GCAS  warning  for  individual  missions  and  across  all 
experimental  conditions.  Of  220  warnings  there  were  208  warnings  that 
elicited  a  control  response  from  the  pilots.  The  12  warnings  that  did 
not  result  in  a  control  input  were  too  brief  for  pilot  response  or  were 
ignored  for  various  reasons  that  will  be  discussed  later.  The  overall 
mean  response  time  was  0.66  seconds  with  a  standard  deviation  of  0.30. 

The  size  of  the  standard  deviation  indicates  a  departure  from  a 
normal  distribution.  In  order  to  determine  the  extent  of  the  departure, 
a  graph  of  the  frequency  distribution  was  drawn  (Figure  11),  and  it  shows 
a  positive  skewness  in  the  distribution  (shaded  area)  of  pilot  response 
times.  Further  inspection  of  these  data  reveals  that  approximately  80 
percent  of  the  area  (10  out  of  13  cases)  in  the  tail  of  the  distribution 
is  from  mission  16  (IMC/unusual  attitudes),  where  the  pilots  were  induced 
into  a  disorienting  environment,  which  no  doubt  contributed  to  the  longer 
and  somewhat  atypical  response  times.  Almost  25  percent  of  the  experi¬ 
mental  runs  on  mission  16  resulted  in  pilot  disorientation  which  was  more 
frequent  than  initially  thought  possible  and  provided  valuable  data  for 
analysis  of  GCAS.  In  order  to  compensate  for  the  skewed  distribution,  the 
median  and  semi-interquartile  range  (Q)  were  computed  to  use  as  a  basis 
for  determining  the  time  allowance  for  pilot  response  time  in  the  GCAS 
algorithm.  In  actuality,  the  difference  between  using  the  mean  and 
median  in  this  case  is  probably  only  of  academic  interest.  If  you  use 
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TABLE  3 


TABLE  SHOWING  PILOT  RESPONSE  TIMES  (SECONDS)  FOR  STICK  INPUT 

FOLLOWING  GCAS  WARNING 


MISSION 

MEAN  TIME  (SECONDS) 

S.D. 

N 

11  (LO  LEVEL  NAV) 

0.53 

0.31 

30 

12  (IMC/LOW  SPEED) 

0.62 

0.28 

30 

13  (LO  ALT  TURN) 

0.57 

0.22 

22 

14  (RANGE) 

0.66 

0.26 

54 

15  (LO  LEVEL 

0.64 

0.27 

26 

BOMB /NAV) 

16  (IMC  UNUSUAL 

0.86 

0.39 

46 

ALTITUDES) 

ACROSS  ALL  MISSIONS: 

M  *  0.66 

S.D.  “  0.30 

N  =  208 

TABLE  4 


ANALYSIS  OF  VARIANCE  (ANOVA)  OF  PILOT  RESPONSE  TIME  (SECONDS) 
FROM  GCAS  WARNING  TO  CONTROL  PILOT 


SOURCE 

SUM  SQ 

DEGREES  OF  FREEDOM 

MEAN  SQ 

F-RATIO 

MISSION  NO. 

24.6 

5 

4.92 

5.38 

ERROR 

184.8 

202 

0.91 

TOTAL 

209.4 

207 

★LEVEL  OF  SIGNIFICANCE  <  0.001 
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the  median  to  generate  a  pilot  population  value,  the  recommended  time  to 
allow  for  pilot  response  time  would  be  1.11  seconds  (0.59  median  value 
plus  twice  the  0.2o  Q  value)  versus  1.25  using  the  mean  and  standard 
deviation  (0.66  mean  value  plus  twice  the  0.30  S.D.  value).  The  authors 
recommend  the  1. 25-second  interval  in  this  case  to  allow  for  the 
possibility  of  pilot  disorientation  or  distraction  in  some  situations. 

This  probably  represents  a  good  compromise  between  allowing  the  operator 
some  flexibility  without  a  significant  increase  in  nuisance  warnings. 

In  comparing  the  mean  response  times  in  Table  3,  we  see  that 
missions  11  and  13  yield  the  lowest  average  response  times  (0.53  seconds 
and  0.57  seconds  respectively),  followed  by  missions  12,  14,  and  15  with 
slightly  higher  response  times  (0.62,  0.64  and  0.66  seconds)  and  finally 
mission  16  with  a  relatively  large  mean  response  time  (0.86  seconds). 
Analysis  of  variance  performed  on  these  data  indicate  the  differences  are 
statistically  significant  at  less  than  the  0.001  level  (Table  4).  The 
significance  was  undoubtedly  due  to  mission  16,  which  was  discussed 
earlier. 

The  standard  deviations  for  the  individual  missions  in  Table  3  are 
slightly  larger  than  you  would  get  in  a  normal  distribution  showing 
increased  variability  and  possibly  some  skewdness.  The  distributions  for 
mission  11  through  15  are  symmetrical,  with  mission  16  showing  the  skew 
previously  mentioned. 

Of  the  twelve  warnings  that  did  not  elicit  a  response,  two  were  too 
brief  for  the  pilot  to  respond,  four  were  ignored  because  the  pilot  was 
in  the  midst  of  weapon  delivery,  two  were  coincidental  with  the  pilot's 
recovery,  and  four  warnings  were  considered  invalid. 

A  summary  of  warning  duration  times  is  shown  in  Table  5  and  provides 
some  interesting  insights  on  GCAS.  Mean  warning  duration  was  3.1  seconds 
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with  a  standard  deviation  of  2.34.  Tha  relative  site  of  the  S.D.  is  even 
greater  than  in  the  response  tine  data,  indicating  even  aore  variability 
and  skewdnesa  (Figure  12).  In  this  instance,  the  skew  is  generated  fron 
two  sources  -  mission  12  and  mission  16.  The  reason  for  the  larger  mean 
and  S.D.  on  mission  16  (induced  disorientation)  was  discussed  earlier  and 
requires  no  further  consent. 

Mission  .-.2  was  a  low  speed  maneuvering  flight  under  instrument 
meteorlogical  conditions  (IMC)  and  was  inserted  into  the  study  to  assess 
the  low  speed  characteristic  of  GCAS.  The  extremely  large  mean  5.60- 
second  warning  duration  is  due  to  the  fact  that  all  warning  situations  on 
mission  12  resulted  in  a  crash  and  the  conditions  necessary  to  silence 
the  voice  warnings  were  never  fulfilled.  The  crashes  are  attributed  to 
the  method  of  computing  terrain  rise,  altitude  loss  and  the  resulting 
pilot  response  times.  A  comprehensive  analysis  of  the  algorithm  is 
covered  in  greater  detail  later  in  the  results  section.  It  is  sufficient 
to  say  at  this  point  that  changes  to  the  algorithm  will  be  required  to 
afford  some  protection  in  the  straight  and  level  flight  regime.  One 
final  point  on  mission  12,  an  offline  analysis  of  the  algorithm  showed 
that  slope  of  the  terrain  was  not  a  factor  in  any  of  the  crashes.  The 
Analysis  of  Variance  of  the  warning  duration  data  (Table  6)  verifies  the 
mean  differences  in  tables  are  significant  and  are  obviously  due  to 
missions  12  and  16. 

One  of  the  factors  that  can  be  crutial  to  the  success  of  a  GCAS  is 
how  long  it  takes  a  pilot  to  get  maximum  G's  on  the  aircraft  -  the 
quicker  you  can  get  to  maximum  G,  the  less  altitude  lost.  Table  7 
summarizes  the  mean  time  to  maximum  G  data.  The  pattern  of  these  data  is 
similar  to  measures  discussed  earlier  -  slightly  larger  than  average 
variability,  and  some  skewdness  with  mission  16  accounting  for  most  of 
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TABU  5 


MEAN  WARNING  DURATION  TIMES 


MISSION 

MEAN  TIME  (SECONDS) 

S.D. 

N 

11  (LO  LEVEL) 

1.85 

1.08 

32 

12  ( IMG /LOW  SPEED) 

5.68 

1.39 

31 

13  (LO  ALT  TURN) 

1.50 

0.94 

22 

14  (RANGE) 

2.32 

2.19 

58 

15  (LO  LEVEL 

30MB /NAV) 

2.28 

1.47 

26 

16  (IMC  UNUSUAL 
ATTITUDES) 

4.38 

2.49 

51 

ACROSS  ALL  MISSIONS: 

M  -  3.10 

S.D.  -  2 .'34 

N  -  220 

TABLE  6 


ANALYSIS  OF  VARIANCE  FOR  WARNING  DURATION  TIMES 


SOURCE 

SUM  SQ 

DEGREES  OF  FREEDOM 

MEAN  JSq 

F^RATIO 

MISSION  NO. 

4986 

5 

897 

25.59 

ERROR 

7503 

214 

35 

TOTAL 

11989 

219 

*  LEVEL  OF  SIGNIFICANCE  <  0.001 
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TABLE  7 


MEAN  TIMS  TO  MAXIMUM  G'S 


MIS8ION 

MEAN  TIME (SECONDS) 

S.D. 

N 

11  (LO  LEVEL  NAV) 

1.86 

0.89 

32 

12  (IMC/LOW  SPEED) 

2.73 

1.42 

31 

13  (LO  ALT  TURN) 

2.16 

0.63 

18 

14  (RANGE) 

2.27 

1.14 

56 

15  (LO  LEVEL 

BOMB /NAV) 

2.62 

1.11 

25 

16  (IMC  UNUSUAL 
ATTITUDES) 

3.80 

1.50 

51 

ACROSS  ALL  MISSIONS:  M 

-  2.68 

S.D 

.-  1.39 

N  -  213 

TABLE  8 

ANALYSIS  OF  VARIANCE 

FOR  MEAN  TIME  TO  MAXIMUM 

G'S 

SOURCE  SUM  SQ  DEGREES 

OF  FREEDOM 

MEAN  SQ 

F-RATIO 

MISSION  NO.  991 

5 

198 

13.20 

ERROR  3093 

207 

15 

TOTAL  4084 

212 

*  LEVEL  OF  SIGNIFICANCE  <  0.001 


it.  The  overall  Man  time  to  maxi  nun  G  ia  2.68  atconda  with  a  S.D.  of 
1.39.  Missions  11  through  IS  ara  grouped  with  small  differences  between 
then,  while  niaaion  16  ia  significantly  larger  (Table  8).  We  should  be 
soMvhat  cautious  in  interpretting  'G'  onset  data  in  fighter  simulators, 
since  they  cannot  sinulate  the  physical  sensation  of  anything  above  1  G. 
It  is  possible  that  'G'  onset  My  be  quicker  in  a  simulator  than  in  an 
actual  aircraft.  However,  the  F-16  force  stick  would,  in  the  opinion  of 
the  authors,  more  likely  be  representative  than  a  simulator  with  e 
displacement  stick.  In  any  case,  the  simulator  data  is  probably 
representative  of  the  actual  aircraft,  but  flight  test  data  should  be 
used  to  verify  this. 

Table  9  depicts  mean  maximum  G  across  missions.  The  mean  values 
range  from  2.32  for  mission  12,  to  5.9  for  mission  16.  Unlike  other  data 
distributions,  Mximum  G  standard  deviations  are  quite  small  in  relation 
to  their  respective  means,  reflecting  normal  symmetrical  and  tight 
groupings.  Mean  differences  between  all  these  individual  missions  are 
significant  (Table  10).  The  Mximum  G  data  reflects  the  maneuvers 
required  on  each  mission.  The  relatively  low  G's  on  mission  12  (2.32) 
are  all  the  G's  available  to  the  pilot  on  a  low  speed  zero  flight  path 
angle  mission;  whereas  5.9  G  mean  seen  in  mission  16  is  representative 
of  IMC  aerobatic  mission.  Although  we  must  not  take  simulator  G  data  too 
literally,  the  figures  in  this  case  appear  to  match  those  typically 
encountered  in  actual  aircraft. 

The  final  group  of  flight  parameter  data  analyzed  was  mean  time  to 
zero  flight  path  angle  (FPA)  (Table  11).  The  total  N  in  this  case  was 
somewhat  reduced  since  the  data  from  mission  12  were  not  applicable, 
since  zero  FPA  was  never  achieved.  The  only  significant  data  shown  are 
that  the  mission  16  mean  is  much  larger  than  the  others,  bjwh  because 
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TABLE  9 


MEAN  MAXIMUM  G'S  ACROSS  MISSIONS 


MISSION 

MAXIMUM  G 

S.D. 

N 

11  (LO  LEVEL  NAV) 

3.38 

0.98 

32 

12  (IMC/LOW  SPEED) 

2.32 

0.34 

31 

13  (LO  ALT  TURN) 

5.52 

1.62 

22 

14  (RANGE) 

4.86 

1.81 

58 

15  (LO  LEVEL 

5.15 

1.96 

26 

BOMB /NAV) 

16  (CMC  JN USUAL 

5.90 

1.11 

51 

ATTITUDES) 

ACROSS  ALL  MISSIONS: 

M  -  4.62  S.D. 

-  2.34 

N--  22 

TABLE  10 

ANALYSIS  OF  VARIANCE  FOR  MEAN  MAXIMUM  G'S  ACROSS  MISSIONS 


SOURCE 

SUM  SQ 

DEGREES  OF  FREEDOM 

MEAN  SQ 

F-RATIO 

MISSION  NO. 

3270 

5 

654.0 

32.22 

ERROR 

4350 

214 

20.3 

TOTAL 

7620 

219 

*  LEVEL  OF  SIGNIFICANCE  <  0.001 
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of  Che  complex  maneuvers  and  the  induced  disorientation  (Table  12). 

Almost  30  percent  of  the  GCAS  events  observed  in  the  study  resulted  in 
simulated  crashes.  A  statistical  summary  of  these  crashes  is  shown  in 
Table  13.  Almost  one-half  the  crashes  occurred  on  mission  12  and  were 
due  to  the  way  the  GCAS  algorithm  is  mechanized.  This  will  be  covered  in 
the  Algorithm  Analysis.  Of  the  remaining  33  crashes,  21  occurred  on 
bombing  type  missions  where  the  pilots  tended  to  ignore  the  warning  until 
weapons  delivery  was  completed,  and  six  occurred  during  aerobatics  where 
disorientation  was  a  factor. 

One  reason  pilots  flew  through  the  warnings  involved  how  frequently 
they  occurred  at  an  obviously  high  altitude,  especially  at  the  higher 
dive  angles  where  pilots  were  trying  to  release  at  the  briefed  altitude 
of  2500  feet.  When  warnings  occurred  at  3000  feet  and  above,  they  would 
have  to  make  a  judgement:  was  the  warning  early  or  was  it  caused  by  a 
higher  than  normal  dive  angle?  This  required  time  and  concentration  at  a 
point  in  the  delivery  where  intense  concentration  was  required  as  the 
delivery  solution.  This  problem  is  not  unique  to  the  air-to-ground 
delivery  mission;  in  fact,  it  is  one  faced  by  pilots  on  almost  every  run 
in  an  unfamiliar  target  area.  In  an  operational  sense,  the  tendency  to 
fly  through  a  warning  will  surely  be  less  in  an  aircraft  than  it  was  in  a 
simulator,  and  pilots  will  at  least  take  a  second  look  at  the  situation 
when  the  warnings  occur.  In  this  same  view,  however,  the  system  will 
lead  to  occasional  early  pull-ups  and  subsequent  dry  passes. 

Probably  the  most  significant  problems  were  those  encountered  in 
mission  12,  where  all  warnings  led  to  crashes  and  on  other  missions  where 
rising  terrain  was  encountered  in  a  near  level  flight  attitude.  In  these 
cases,  pilots  received  late  warnings  or  none  at  all.  The  underlying 
causes  of  this  problem  are  described  in  Section  3,  Algorithm  Analysis 
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TABLE  11 


MEAN  TIME  TO  ZERO  FLIGHT  PATH  ANGLE 


MISSION 

MEAN 

TIME 

(SECONDS) 

S.D. 

N 

11  (LO  LEVEL  NAV) 

2.06 

1.19 

22 

12  (IMC/LOW  SPEED) 

N/A 

N/A 

N/A 

13  (LO  ALT  TURN) 

2.36 

0.79 

22 

14  (RANGE) 

2.72 

1.30 

38 

15  (LO  LEVEL 

BOMB /NAV) 

2.73 

1.73 

21 

16  (IMC  UNUSUAL 
ATTITUDES) 

7.20 

1.73 

43 

ACROSS  ALL  MISSIONS: 

M  - 

3.88 

S.D. 

-  2.56 

N  -  146 

TABLE  12 

ANALYSIS  OF  VARIANCE  MEAN  TIME  TO  ZERO  FLIGHT  PATH  ANGLE 


SOURCE 

SUM  SQ 

DEGREES  OF  FREEDOM 

MEAN  SQ 

F-RATIO 

MISSION  NO. 

67650 

4 

16912 

89.7 

ERROR 

26583 

141 

188 

TOTAL 

94233 

145 

*  LEVEL  OF  SIGNIFICANCE  <  0.001 
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TABLE  13 


SUMMARY  OF  CRASH  DATA 


NO.  CRASHES 

NO.  WARNINGS 

PERCENT 

OF  MISSION 

PERCENT 
OF  TOTAL 

MISSION  11 

5 

32 

16 

8 

MISSION  12 

31 

31 

100 

48 

MISSION  13 

1 

22 

5 

2 

MISSION  14 

14 

58 

24 

22 

MISSION  L5 

7 

26 

27 

11 

MISSION  16 

6 

51 

12 

9 

TOTAL: 

64 

220 

N/4 

100 

38 


TABLE  11 


MEAN  TIME  TO  ZERO  FLIGHT  PATH  ANGLE 


MISSION 

MEAN  TIME  (SECONDS) 

S.D. 

N 

11  (LO  LEVEL  NAV) 

2.06 

1.19 

22 

12  (IMC/LOW  SPEED) 

N/A 

N/A 

N/A 

13  (LO  ALT  TURN) 

2.36 

0.79 

22 

14  (RANGE) 

2.72 

1.30 

38 

15  (LO  LEVEL 

2.73 

1.73 

21 

BOMB /NAV) 

\ 

16  (IMC  UNUSUAL 

7.20 

1.73 

43 

ATTITUDES) 

ACROSS  ALL  MISSIONS: 

M  -  3.88  S.D. 

-  2.56 

N  “  146 

TABLE  12 

V 

ANALYSIS  OF  VARIANCE  MEAN  TIME  TO  ZERO  FLIGHT  PATH  ANGLE 

\ 

SOURCE  SUM  SQ  DEGREES  OF  FREEDOM  MEAN  SQ  F-RATIO 


MISSION 

NO.  67650 

4 

16912 

ERROR 

26583 

141 

188 

TOTAL 

94233 

145 

*  LEVEL 

OF  SIGNIFICANCE  <  0.001 
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TABLE  13 


SUMMARY 

OF  CRASH  DATA 

NO.  CRASHES 

NO.  WARNINGS 

PERCENT 

OF  MISSION 

PERCENT 
OF  TOTAL 

MISSION 

11 

5 

32 

16 

8 

MISSION 

12 

31 

31 

100 

48 

MISSION 

13 

1 

22 

5 

2 

MISSION 

14 

14 

58 

24 

22 

MISSION 

15 

7 

26 

27 

It 

MISSION 

16 

6 

51 

12 

9 

TOTAL: 

54 

220 

N/A 

100 

(in  3. a  Condition  1  and  3.c).  Basically,  the  problem  stems  from  system 
inhibits  installed  to  prevent  nuisance  warnings  during  ridge  crossings 
and  it  is  one  which  will  be  extremely  difficult  to  solve.  If  the 
inhibitcf  are  removed  to  provide  a  "look  ahead"  or  terrain  projection 
capability,  nuisance  warnings  go  down;  if  the  inhibits  are  used,  there 
is  little  or  no  protection  against  rising  terrain. 

2.  QUESTIONNAIRE  DATA 

Pilot  ratings  on  all  aspects  of  operational  utility  were 
overwhelmingly  favorable  towards  GCAS.  These  ratings  are  summarized 
in  Table  14.  Eight  out  of  ten  pilots  thought  the  warnings  were  timely 
and  the  system  was  either  useful  or  extremely  useful  in  terms  of  system 
utility.  These  same  pilots  also  said  they  got  nuisance  warnings 
occasionally,  did  not  tend  to  over  rely  on  the  warnings,  and  missed  no 
warnings.  All  participants  agreed  the  attention  getting  value  of  GCAS 
was  good.  Although  all  pilots  thought  the  system  was  either  good  or 
satisfactory,  they  all  agreed  some  improvements  were  required. 

In  the  way  of  system  improvement,  there  was  general  agreement  (6 
out  of  10)  that  ridge  crossings  presented  a  problem  and  require  an 
algorithm  to  modify  and  improve  the  GCAS  in  this  situation.  Two  pilots 
reported  that  they  would  like  a  system  with  some  foward  looking 
capability.  Some  sort  of  anticipatory  cue  prior  to  the  warning  was 
suggested  by  three  of  the  pilots.  Finally,  one  pilot  suggested  a  "more 
timely"  voice. 

General  comments  made  by  the  pilots  were  all  favorable  and  two 
thought  GCAS  should  be  installed  immediately.  Six  out  of  ten  pilots 
thought  the  slope  warning  unnecessary,  two  thought  it  moderately  useful, 
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A 


one  preferred  it,  end  one  made  no  comment.  Finally,  the  pilots  generally 
agreed  that  the  simulation  was  satisfactory  for  rating  GCAS,  tf  the 
reader  is  interested  in  reading  all  pilot  comments,  they  are  provided  in 
the  Appendix. 
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TABLE  14 


QUESTIONNAIRE  DATA  SUMMARY 


TIMELINESS: 


2/10  SLIGHTLY  EARLY 
7/10  JUST  RIGHT 
1/10  SLIGHTLY  LATE 


ATTENTION  GETTING  VALUE: 

5/10  ALL  THE  TIME 
5/10  MOST  OF  THE  TIME 


NUISANCE  WARNING  OCCURRENCE: 

8/10  OCCASIONALLY 
2/10  MOST  OF  THE  TIME 


MISSED  WARNINGS: 

8/10  NONE 

2/10  YES 
* 


SYSTEM  UTILITY: 

8/10  USEFUL  OR  EXTREMELY  USEFUL 
2/10  MODERATE  USEFULNESS 


OVER  RELIANCE  ON  WARNING: 

8/10  SELDOM  OR  NEVER 
1/10  OCCASIONALLY 
1/10  FREQUENTLY 


ADEQUACY /SUITABILITY : 

8/10  SATISFACTORY,  BUT  NEEDS  IMPROVEMENT 
2/10  GOOD  OR  VERY  GOOD 


3.  ALGORITHM  ANALYSIS 


The  GCAS  algorithm  was  analysed  by  CSDF  in  order  to  determine  the 
validity  of  warnings  in  various  flight  conditions.  For  the  purpose  of 
this  evaluation,  an  invalid  warning  is  one  which  occurs  too  early  (a 
"nuisance"  warning),  or  one  which  occurs  too  late  for  a  safe  recovery. 

The  algorithm  is  briefly  explained  here  in  order  to  provide  a  background 
for  the  analysis.  Cubic's  algorithm  continuously  compares  the  aircraft's 
present  height  above  the  ground  with  the  altitude  required  for  dive 
recovery.  The  algorithm  predicts  altitude  loss  during  dive  recovery  as  a 
sum  of  the  following: 

a.  Altitude  loss  due  to  terrain  rise  during  the  pull-up. 

b.  Altitude  loss  due  to  pilot  response  time. 

c.  Altitude  loss  during  roll  recovery. 

d.  Altitude  loss  during  a  5-G  pull-up. 
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The  following  paragraphs  describe  esch  portion  of  the  altitude  loss 
calculation  in  detail.  Reasons  for  early  or  late  warnings  are  also 
discussed. 

a.  Altitude  loss-pilot  response 

The  initial  portion  of  the  altitude  loss  is  fairly  simple  to 
compute.  During  the  period  of  time  before  the  pilot  has  made  any 
response  to  a  warning,  the  aircraft  is  projected  to  continue  its  present 
vertical  trajectory.  The  current  vertical  velocity  is  multiplied  by  a 
computed  response  time  (T^^)  to  calculate  altitude  that  would  be  lost 
from  warning  to  initial  stick  input. 

ROTE:  If  the  altitude  loss  is  negative  (i.e.,  the  airplane  clin&ing  the 

absolute  value  of  the  altitude  loss  is  used),  Cubic  assumed  a  basic 

response  time  of  two  seconds  and  modified  it  by  several  conditions: 

Condition  1  :  Small  Dive  Climb  Angle  (GAMMA) 

In  order  to  reduce  the  possibility  of  early  warnings  during  ridge 

crossings  T  is  reduced  when  the  aircraft  is  in  level  or  near  level 
resp 

flight.  The  following  relation  is  used: 

If  \  <  5* 


T 

resp 


0.4 


So  at  zero  flight  path  angle,  ■  0 

Condition  2  :  Roll  Angle/Roll  Rate  (0,  )  is  assumed  that  if  the 

pilot  is  rolling  out  of  a  bank  at  the  time  a  warring  occurs,  his  reaction 

time  will  be  less,  so: 

* 

If  0  >  15  deg/sec 


and 

&  and  H  are  of  opposite  sign, 


T 

resp 


*  0.3  sec 
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NOTE:  This  condition  overrides  Condition  1. 

Condition  3  :  Vortical  Velocity,  Vertical  Acceleration.  Response 

tine  ia  assumed  to  be  shorter  when  the  pilot  is  already  reducing  his 
downward  velocity.  So,  whenever  the  aircraft  has  a  downward  flight  path 
<Vt  <0)  end  a  positive  acceleration  vertical  acceleration  (A^O),  the 
response  tine  used  in  calculating  the  airplane's  future  flight  path  angle  is 
reduced  by  half.  Otherwise,  if  the  airplane  has  a  downward  acceleration, 
its  vertical  velocity  will  be  modified  using  full  response  time  as  calculated 
by  Conditions  1  and  2.  So  the  extrapolated  dive/climb  angle,  %  is 

6  XU 

calculated  as  follows: 


where  vx(ftxt)  "  va  +  Aa^Text"  extr,P°lated  vertical  velocity) 

T  .  •  T  (modified  by  Condition  3) 

ext  reap 

•  Horiaontal  Velocity 

The  value  Hext  is  used  in  computing  the  altitude  required  for  a  5-G 
(cr  available  G)  dive  recovery  (Paragraph  C).  If  the  airplane  has  a 
positive  vertical  velocity  and  a  positive  vertical  acceleration,  the 
extrapolation  is  not  performed.  Additionally,  if  Xext  ^  0>  no 
(positive)  altitude  change  will  be  calculated  due  to  pilot  response 
time. 

Condition  4  :  Previous  warning  within  3  seconds.  It  is  assumed  that 
if  a  pilot  has  recently  (within  the  past  3  seconds)  experienced  a  pull-up 
warning,  he  will  react  to  subsequent  warnings  more  quickly.  Thus,  if  the 
warning  is  not  on  and  less  than  3  seconds  have  elapsed  since  the  last 
warning  ceased, 

T  ■  1.5  seconds 
reap 

Condition  5  :  Warning  in  progress.  When  a  pull-up  warning  is  in 
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effect,  the  response  time  it  reduced  by  one  elite  step  etch  Cine  the 
celculetion  it  performed. 

So,  for  the  betic  2  tecond  retponte  tine,  T  will  be  reduced  to  sero 
efter  2  tecondt  of  warning. 

1. 1  Problems 

a. 1*1  Early  warning! 

a. 1.1.1  The  baaic  retponte  time  of  2  tecondt  is  too  long.  A 

more  realistic  baseline  would  be  between  1  and  1.25  seconds  (see 

performance  section  of  results). 

a. 1.1.2  The  establishment  of  a  0.3  second  response  time  under 

Condition  2  is  intended  to  reduce  nuisance  warnings.  In  some  cases, 

however,  it  actually  causes  an  increase  in  invalid  warnings.  Consider 

first  the  case  where  a  pilot  gets  a  warning  during  descending  turn 

(  f  >5*).  His  response  time  (T  .  )  starts  at  a  value  of  2  seconds. 

resp 

As  he  begins  to  roll  out  of  his  turn,  Condition  2  immediately  reduces 
his  response  time  to  0.3  seconds.  This  will  often  cause  the  alarm  to 
terminate  since  altitude  loss  due  to  response  time  has  decreased. 

However,  as  soon  as  (a)  his  roll  rate  falls  below  15  degrees  per  second 
or  (b)  his  bank  angle  changes  sign  because  he  overshoots  wings  level, 
the  response  time  resets  to  1.5  seconds  (Condition  4).  This  can  cause 
the  warning  to  come  back  on  even  though  he  has  begun  a  satisfactory 
recovery.  Modifications  to  altitude  loss  based  on  roll  rate  would  perhaps 
be  more  appropriately  applied  to  the  roll  recovery  portion  of  the 
algorithm. 

It  would  also  be  prudent  to  suppress  response  time  reset  (Condition  4) 
completely  for  3.0  seconds  after  a  warning  has  been  satisfied.  As  a 
minimum,  it  should  be  reset  to  a  much  smaller  value  than  1.5  seconds. 
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a. 1.1.3  It  ia  not  valid  to  assume,  aa  in  condition  1,  that  a 
pilot'a  response  tin#  ia  laaa  in  laval  (.%  ■  0)  flight.  Tha  raaponaa 
times  demonstrated  in  thia  study  showed  no  substantial  raduction  under 
thasa  conditions.  In  laval  flight,  tha  only  aourca  of  a  ground 
collision  can  ba  ground  riaa.  Paragraph  a.  shows  that  thia 
algorithm  does  not  conaidar  pilot  raaponaa  time  in  calculating  altitude 
loss  due  to  terrain  riaa.  Therefore,  Condition  1  provides  only  a 
negligible  reduction  in  nuisance  warnings  during  ridge  crossings. 

a. 1.1.4  Condition  3  modifies  the  airplane's  predicted 
altitude  loss  according  to  instantaneous  vertical  velocity  and 
acceleration.  It  can  cause  abrupt  changes  in  the  predicted  altitude 
loss  as  the  aircraft  transitions  from  one  set  of  conditions  to  another, 
resulting  in  erroneous  warning  resets  in  a  manner  similar  to  the 
example  in  paragraph  al.1.2.  Also,  since  the  flight  path  is  never 
extrapolated  upward  during  a  climbing  maneuver,  some  needless  warnings 
may  occur  (Figure  14). 

Consider  the  case  where  a  pilot  is  pulling  up  to  avoid  a 
ridgeline.  The  algorithm  projects  aero  altitude  gain  during  the  response 
time.  So  even  though  the  pilot  may  be  climbing  and  accelerating  upward, 
he  will  get  a  warning  based  on  the  fact  that  his  current  altitude  is  less 
than  the  projected  terrain  height.  This  is  compounded  by  the  fact  that  the 
time  used  for  calculating  terrain  rise  is  based  on  dive  recovery  time.  A 
positive  dive  recovery  time  will  be  calculated  even  though  the  airplane's 
flight  path  is  above  the  projected  terrain  (Paragraph  d). 
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Figure  14.  Possible  Nuisance  Warning  Caused  by 

Failure  Co  Project  Flight  Path  Upward 

A  more  accurate  way  to  approximate  the  altitude  loss  during  pilot 

response  time  would  be  to  simply  apply  the  following  equation  to  obtain 

altitude  lost  during  pilot  response.  That  is: 

ft 


A  Z 


reap 


-V  Cos  (  V  ) 

5 


where: 

«  Present  flight  path  angle 
* 

V  _  »  V ,  +  Y  (t  ) 

'2  1  1  resp 

V  *  Aircraft  Velocity 

and,  assuming  relatively  small  change  in  V  , 

e 

^  ,  ■  g  (n  -  cos  Y.  ) 

v  v 

nv  ■  n  cos  tf) 

This  simple  equation  provides  both  altitude  loss  (or  gain)  during  pilot 
reaction  time  and  a  predicted  flight  path  angle  to  be  used  in  the  dive 
recovery  equations.  It  eliminates  abrupt  changes  in  predicted  loss  and 
resultant  nuisance  warnings  that  are  characteristic  of  the  current 
algorithm. 


b.  Altitude  Loss  During  Roll  Recovery 

The  GCAS  algorithm  predicts  the  altitude  that  will  be  lost  while 
the  pilot  rolls  wings  level.  This  calculation  is  based  on  the  assumption 
that  the  pilot  will  not  begin  his  dive  recovery  until  he  has  brought  his 
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wings  nearly  level.  The  calculation  is  made  in  the  following  manner: 


(1)  The  time  for  roll  recovery  is  calculated. 


Where  0  *  roll  angle 
fi  pred  ■  predicted  roll  rate 

-  70  *  /Sec  for  20  *  <  0  <  85  ° 

56  *  /Sec  for  85  *  <  0  <145  # 

90  *  /Sec  for  145  *<  0  <180  * 


b.l  Problems 

b.1.1  Early  warnings 

b.l. 1.1  The  algorithm  assumes  incorrectly  that  no  dive 
recovery  begins  until  roll  recovery  is  complete.  In  fact,  test  results 
showed  that  pilots  reduced  bank  and  applied  recovery  "G"  simultaneously. 

In  cases  where  the  aircraft  is  in  a  high  "G"  turn  merely  rolling  out  of 
the  bank  while  maintaining  the  "G"  affected  most  recoveries. 

b.l. 1.2  The  use  of  three  distinct  roll  rates  for 
predicting  roll  recovery  times  causes  abrupt  changes  in  the  altitude  loss 
prediction.  For  instance,  the  algorithm  would  predict  a  Tfr  of  0.91 
seconds  for  a  bank  angle  of  84’.  By  increasing  the  bank  to  85°,  T 
jumps  to  1.2  seconds.  Decreasing  the  bank  from  145°  to  144“  causes  T^ 
to  change  from  2.2  seconds  to  1.4  seconds.  At  450  knots  and  30*  nose 
down  the  latter  time  difference  (0.8  seconds)  is  equivalent  to  an  altitude 
loss  of  about  300  feet.  At  450  knots  and  30*  nose  down,  the  latter  time 
difference  (0.8)  is  equivalent  to  an  attitude  loss  of  about  300  feet.  A 
continuous  function  for  calculating  T  from  would  eliminate  the 
abrupt  changes  and  the  resultant  early  warnings. 

b.1.2  Late  warnings.  No  late  warnings  were  associated 
with  the  roll  recovery  calculations. 

c.  Altitude  Loss  During  Dive  Recovery 

The  algorithm  calculates  the  altitude  loss  that  would  occur 
during  a  wings  level,  constant  G  dive  recovery  in  the  following  manner: 

(1)  The  basic  equation  for  a  constant  G  constant  true  airspeed  dive 
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recovery  is: 


WhereAZpU  ■  the  altitude  lost  during  the  pullup 

V  ■  aircraft  true  airspeed  (ft/sec) 

N  •  G  available  (nax  of  5) 
y  ■  extrapolated  flight  path  angle 
g  ■  32.2  ft/sec  2 

This  equation  is  based  on  the  assumption  that  dive 
recovery  is  complete  when  Y  ■  0*  . 

(2)  The  value  for  n  was  determined  from  a  look-up  table  using  the 
following  piecewise  linear  schedule  (Figure  15). 

n  (o  ») 


Figure  15  G-Available  vs  Indicated  Airspeed 

(3)  Since  the  constant  G  dive  recovery  equation  does  not  account  for 
altitude  lost  during  application  of  G,  it  was  modified,  resulting  in  the 
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following  equation: 

r  r 

Al  -  K  V2  In  i  n,  -  cos 

pu  2  ext 

where  ■  n  for  n  <  2.2 

-  n/2  for  n  >2.2 
K  -  .jV,  co.  Y  |U  ♦  .2co.  Y  eJt 

♦  «3*i  ♦  \ 

V  ■  true  airspeed 
Vj*  indicated  airspeed 

The  constants  a^  were  derived  from  data  obtained  by  Cubic  during  a 

limited  series  of  controlled  dive  recoveries  in  instrumented  aircraft. 

The  equation  for  A  Z  was  assumed  to  be  of  the  above  form  and  a  curve 

pu 

fit  was  performed  to  determine  values  for  constants  a^  for  five 
distinct  airspeed  regimes, 
c.l  Problems 
c.1.1  Early  warnings 

c.l.  1.1  The  equation  for  ^  Z^,  along  with  its  variables 
a^  and  K,  was  derived  based  on  the  assumption  that  the  pilot  will  start 
his  dive  recovery  from  a  load  factor  of  1  G.  Even  during  a  5-G  dive 
recovery,  the  algorithm  continues  to  predict  altitude  loss  based  on  this 
equation  which  (as  explained  in  paragraph  c  (3)  above)  has  been  tailored 
to  include  altitude  loss  during  the  buildup  from  1  to  3  G's.  This  can 
cause  early  warnings  under  certain  conditions.  Other  times,  the  warning 
will  stay  on  after  the  aircraft  is  no  longer  in  danger.  The  designers  of 
the  algorithm  have  compensated  for  this  inflexibility  somewhat  by 
reducing  the  altitude  loss  due  to  pilot  response  time  (Paragraph  a.) 
when  the  aircraft  has  an  upward  acceleration,  but  this  does  not  adequately 


solve  the  problem  under  all  conditions. 
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c. 1.1.2  The  dive  recovery  equation  calculate*  altitude 
loaa  in  reducing  Y  to  aero.  In  caeea  where  the  terrain  i*  rising,  this 
results  in  an  excessive  estimate. 


Figure  16,  Error  Caused  by  Estimating  Az 
Based  on  Recovery  toY*  0  p 


Figure  16  shows  an  aircraft  in  a  dive  recovery  in  the  presence  of 
sloping  terrain.  The  aircraft  is  nearest  the  terrain  at  the  point  where 
the  flight  path  angle  Y  ,  is  equal  to  the  angular  slope  of  the  terrain. 
This  altitude  H^,  is  higher  than  the  altitude  where  Y  “O,  here 
represented  by  Hj.  By  calculating  altitude  lost  based  on  reducing Y  to 
zero,  an  error  of  magnitude  (H^  -  H., )  is  introduced,  resulting  in  an 
early  warning. 

This  is  an  important  consideration  in  limiting  nuisance  warnings  due 
to  rising  terrain.  In  level  flight,  approaching  rising  terrain,  calcula¬ 
tions  based  on  matching  to  the  terrain  slope  should  result  in  a  negative 


c.1.2  Late  warnings 

No  late  warnings  were  noted.  However,  recommend  that  the 
values  for  K  and  a^,  if  used,  be  verified  through  more  extensive 
experimentation. 


d.  Altitude  loss  due  to  terrain  rise 

The  algorithm  calculates  the  altitude  loss  due  to  ground  rise 
using  a  filtered  estimation  of  the  terrain  slope  directly  below  the  air 
craft.  To  estimate  the  change  in  terrain  height  during  recovery,  the 
slope  is  extrapolated  for  a  time  T  which  is  calculated  as  a  function 
of  initial  flight  path  ^angle.  The  formula  for  Tjf  is: 

T 

gr 


2V 

-1 

tan 

r  gr 

|(n2r 

1) 

t  ' 

tan  2 

•M 

| 

n  -  1 

• 

where: 


T  *  time  used  to  calculate  ground  rise 

gr 

V  ■  aircraft  velocity  (ft/sec) 
g  M  32.2  ft/sec 
n  ■  4  (G '  s ) 

^ *  aircraft  flight  path  angle 

Once  T^r  has  been  determined,  the  following  checks  are  made  to 
determine  whether  or  not  to  calculate  ground  rise. 


1.  Calculate  T. 

impact 


T . 

impact 


Height 


V  (terrain)  -  V  (aircraft) 
z  z 


where  T , 

impact 


Height 


the  time  it  will  take  the  aircraft  to 

hit  the  terrain  based  on  its  current  vertical 
velocity  ^vz(aircraf t )  ^and  clie  current  rate  of 

ground  rU.  <»,(M„,in)> 

■  current  height  above  terrain 
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d.l.  Other  problems 

Assuming  that  the  calculated  terrain  slope  is  accurate  and  the 
terrain  is  absolutely  consistent,  the  most  accurate  way  to  calculate  ground 
rise  is  to  multiply  the  slope  of  the  terrain  by  the  horizontal  distance  the 
airplane  will  travel  during  reaction  time,  roll  recovery,  and  dive  recovery 
(to  match  Y  to  terrain  slope).  The  algorithm  calculates  terrain  rise  based 
exclusively  on  the  time  it  takes  to  perform  a  constant  4-G  dive  recovery  to 
zero  flight  path  angle.  This  involves  several  inaccuracies: 


(1)  V  ,  .  . 

z (terrain) 


(ft/sec)  must  be  calculated  based  on  the  instantaneous 


horizontal  velocity  of  the  aircraft  dx/dt  and  the  slope  dh/dx  of 
the  local  terrain,  that  i3: 


V 

z 


(terrain) 


dh 

dx 


dx 

dt 


Since  dx_  increases  during  the  dive  recovery,  as  the  aircraft 
dt 


velocity  vector  nears  horizontal,  ^z(terra£n)8h°uM  increase.  Basing  the 
terrain  rise  on  this  initial  value  of  the  aircraft's  horizontal  velocity 
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will  result  in  an  underestimation  of  terrain  rise. 

(2)  The  distance  traveled  during  reaction  time,  roll  recovery,  and 
application  of  recovery  G  is  not  considered.  The  net  effect  of  these 
omissions  is  an  additional  negative  error  (underestimate)  of  the  terrain 
rise. 

(3)  The  calculation  of  T  is  based  on  reducingV to  sero.  Thus,  if 
the  airplane  is  in  level  flight,  approaching  rising  terrain  T  andAZ^ 
are  sero,  affording  absolutely  no  protection  against  rising  terrain.  If 
the  "dive  recovery"  portion  of  the  ground  rise  calculation  considered  the 
distance  traveled  while  matching  Y  to  the  slope  angle  of  the  terrain, 
more  protection  would  be  provided. 


(4)  From  a  cursory  mathematical  analysis  of  the  equation  for  T^, 

2 

it  appears  that  there  may  be  a  sign  error  in  the  terms  n  +1. 


This  equation  appears  to  be  based  on  the  following  relation  for 


constant  -  g  dive  recoveries: 

if  ■  g  .  (n  -  cos  V  ) 
dt  V 


dt  ■  vdY _ 

gCn-cos  V*  ) 


Integrating  both  sides  of  this  equation  gives: 


0  g(n-cos 


r) 


(Ref:  CRC  Standard  Mathematical  Tables,  Fifteenth  Edition,  Integral  No. 260) 

I  2~  f  2 

Note  the  terms y  n  -  1  in  place  of  v n  +1. 


This  introduces  an  error  as  does  the  use  of  4G’s  instead  of  5G's  (or  G 
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available)  for  tha  quantity  n 


(S)  Since  the  algorithm  takes  the  absolute  value  of  the 
quantity 

r 


tan 


-1 


n2  +  1 


tan  j 


n  -  1 


a  positive  value  will  be  returned  for  T  whether  the  aircraft  is 
climbing  or  diving.  Thus,  an  airplane  in  a  slight  climb,  well 
clear  of  rising  terrain  will  receive  a  nuisance  warning,  where 
an  aircraft  in  level  flight,  in  danger  of  striking  that  same 
terrain  will  receive  no  warning  at  all  (Figure  17). 


(a)  (b) 

This  aircraft  receives  a  This  aircraft  receives  no 

nuisance  warning.  warning  at  all. 

Figure  17 

(6)  Since  the  algorithm  uses  a  linear  function  to  project  the  change 
in  terrain  elevation,  early  warnings  occur  whenever  the  first 
deviative  of  the  terrain  slope  is  negative  (Figure  18). 
Basically,  the  linear  predictor  is  too  steep  for  ups  lopes  and 
too  shallow  for  downs  lopes. 


Figure  18 


(b) 

Too  steep  for  ups  lope 


(a) 


Too  shallow  for  downs  lope 


Because  of  previously  discussed  characteristics  of  the  GCAS 
algorithm,  very  few  warnings  occurred  as  the  aircraft  approached  rising 
terrain.  However,  when  a  "bunting"  maneuver  was  performed  (a  slight 
negative-G  pushover)  to  descend  on  the  back  side  of  a  ridgeline,  numerous 
false  warnings  were  experienced.  It  is  possible  that  some  of  the  false 
warnings  could  be  eliminated  if  a  modified  second  order  curve  were  used 
to  predict  terrain  elevation. 


lAfg  WARNINGS 


RISING  TERRAIN,  LONG  STEER  TURN 

•  CARA  BEYOND  LIMITS 

•  AIRCRAFT  DOES  NOT  ISEE* 
RISING  TERRAIN 


,1 

UPO 


INVALID  WARNINGS  OCCUR  IN  LONG, 
DESCENDING  TURNS 


LAST  CARA 
UPDATE 


Figure  20  Nuisance  Warnings  —  Terrain  Extrapolation 
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4 


TERRAIN  ESTIMATION 


The  GCAS  algorithm  is  designed  for  use  in  conjunction  with  the 
Combined  Altitude  RADAR  Altimeter  (CARA),  which  is  specified  to  provide 
accurate  altitude  whenever  the  aircraft  bank  and  pitch  angles  are  less 
than  +60*  and  +45*  respectively.  No  forward  looking  sensor  is  avail¬ 
able  to  measure  terrain  height  along  the  predicted  flight  path.  A 
Kalman  Filter  provides  an  estimate  of  terrain  elevation  and  terrain  slope 
beneath  the  aircraft.  When  the  CARA  inputs  are  unavailable,  the  algorithm 
retains  the  last  estimated  terrain  altitude  and  extrapolates  it  according 
to  its  last  estimated  slope  for  a  period  of  time,  T  ,  which  varies 
according  to  aircraft  flight  parameters  and  terrain  consistency.  When  this 
time  period  has  expired,  the  extrapolation  is  discontinued  and  the  last 
terrain  height  estimate  is  held  until  CARA  inputs  are  restored. 

This  approach  allows  the  GCAS  to  continue  to  provide  protection  over 
a  much  larger  range  of  flight  maneuvers  than  most  alternative  systems, 
which  merely  shut  down  when  a  current  sensor  input  is  unavailable. 

However,  some  problems  were  encountered  during  the  simulation  which 
demonstrated  limitations  to  the  effectiveness  of  the  terrain  extra¬ 
polation/hold  feature. 

The  basis  for  all  the  limitations  is  the  fact  that  terrain  is,  to  a 
large  degree,  unpredictable.  If,  as  in  Figure  19,  for  instance,  the  air 
craft  is  in  a  long,  high-banked  (>60*)  turn,  and  encounters  abruptly 
rising  terrain,  no  warning  will  be  provided.  If  the  aircraft  is  in  a 
descending  turn  over  an  abrupt  drop  off,  such  as  a  ridge  (Figure  20),  an 
invalid  warning  will  occur  as  the  aircraft  approaches  the  estimated 
terrain  altitude.  These  faults  are  more  or  less  unavoidable  as  long  as  a 
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bank  limited  tensor  with  no  forward-looking  capability  ia  used.  Overall, 
it  appears  that  in  apite  of  the  shortcomings,  the  terrain  hold /extrapola¬ 
tion  feature  ia  a  worthwhile  addition  to  the  GCAS.  In  coopariaion  to 
syateas  with  no  extrapolation  feature,  GCAS: 

1.  Provides  increased  protection  during  large  amplitude  maneuvering 
over  flat  or  regularly  sloping  terrain. 

2.  Increases  the  nuofcer  of  nuisance  warnings  slightly  due  to  turning 
descents  over  irregular  terrain. 

3.  Provides  the  same  protection  against  steep  turns  into  rising 
terrain — none. 
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5. 


RECOMMENDATIONS 


There  is  no  question  that  Cubic's  approach  to  the  GCAS  process  is 
most  ambitious  and  promising.  The  concept  of  terrain  estimation,  both  for 
periods  when  the  sensor  inputs  are  degraded  and  for  prediction  of  altered 
loss  due  to  ups’ope,  may  give  GCAS  a  capability  unmatched  by  any  system 
using  similarly  limited  sensors. 

Recommend,  however,  that  the  preceding  analysis  be  considered  carefully 
prior  to  implementation  of  the  system.  Follow-on  testing  in  actual  air¬ 
craft  should  be  used  to  evaluate  the  Kalman  Filter  and  its  capability  with 
a  CARA  system  over  actual  terrain.  These  portions  of  the  system  could  not 
be  properly  evaluated  in  the  simulator  due  to  (1)  dissimilarities  between 
simulator  terrain  data  base  and  actual  terrain  and  C2)  uncertainties  about 
the  true  characteristics  of  CARA.  If  the  limitations  mentioned  herein  are 
adequately  remedied,  then  the  generic  GCAS  stands  a  very  good  chance  of 
saving  valuable  aircraft  and  irreplaceable  crews. 
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6.  RESULTS 


\ 


In  order  to  integrate  the  pilot  performance/response  time  data  with 
the  GCAS  algorithm  analysis  and  pilot  questionnaire,  a  brief  summary  is 
appropriate.  First,  the  duration  of  the  GCAS  warning  varied  depending  on 
type  of  mission  flown.  The  GCAS  duration  ranged  from  1.85  seconds  for 
low  level  navigation  to  4.38  seconds  for  IMC  unusual  attitudes.  The 
average  warning  duration  across  all  missions  was  3.1  seconds 
(S.D.  ■  2.34).  As  is  the  case  with  most  response  time  data,  it  tends  to 

be  slightly  skewed  due  to  a  variety  of  factors  that  were  discussed. 

The  average  response  time  to  a  GCAS  warning  is  0.66  seconds  with  a 
standard  deviation  of  0.30.  Taking  all  factors  discussed  into  account 
and  adding  2  sigma  to  the  mean  response  time,  the  recommended  allowance 

for  pilot  response  in  the  GCAS  is  2  seconds.  Performance  data  were  also 

gathered  in  time  to  maxiuum  G  (M  *  2.68;  S.D.  ■  1.39),  maximum  G's  pulled 
(M  *  4.62;  S.D.  ■  2.34),  and  time  to  zero  flight  path  angle  (M  -  3.88; 
S.D.  -  2.56). 

The  Algorithm  Analysis  indicates  some  modifications  are  necessary 
including  the  following: 

1)  Modifications  for  smoothing  roll,  roll  rate  and  acceleration. 

2)  Shortening  flight  path  extrapolation  for  negative  G  "bunts." 

3)  Adjusting  roll  recovery  rates  and  varying  them  with  airspeed  and 
bank. 

4)  Modifying  method  of  computing  terrain  rise. 

5)  Modifying  reaction  time  allowance  at  zero  flight  path  angle. 

Finally,  the  questionnaire  data  indicated  a  good  pilot  acceptance  and 

the  results  indicate  that  GCAS  is  a  viable  system  for  partial  solution  of 
CFIT . 
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SECTION  IV 


CONCLUSIONS 

A  generic  GCAS  appears  to  be  a  useful  system  that  has  good  pilot 
acceptance,  but  needs  some  improvements  prior  to  implementation. 
The  recommended  improvement  to  the  algorithm  should  be 
incorporated  and  verified  in  simulation  (see  recommendations). 
Flight  test  the  improved  system. 


GROUND  COLLISION  AVOIDANCE  SYSTEM  PILOT  QUESTIONNAIRE 


A 


INSTRUCTIONS:  The  purpose  of  this  questionnaire  is  to  obtain  information 
from  you  about  your  previous  flying  experience*  Your  answers  to  these 
questions  will  help  us  in  our  evaluation  of  this  simulation.  Your  honest 
opinions  are,  therefore,  essential  and  win  be  kept  confidential. 

If  you  have  any  questions,  please  ask  the  questionnaire  administrator  for 
assistance.  Take  as  much  time  as  necessary  to  answer  the  questionnaire. 

PERSONAL  DATA: 

Name  (last,  first,  mi): 

Rank:  _ 

Duty  AFSC:  _ 

Organization  and  Symbol:  _____ ___________________ ________ _____ _____ 

Duty  Station:  _ 

Duty  Phone:  ' 

Wing  Commander,  Squadron  Commander,  DO:  _ _ 

Aero  Rating:  _ _ 

Age:- _ 

Height:  _____________________________________________ _________________________ 

Weight:  _ 

Vision,  Corrected:  _  Uncorrected:  _ 

Years  in  Military  Service :_ _ 

What  type(s)  of  aircraft  have  you  flown?  (List) 

Aircraft:  _ 

Fighter  hours:  _ _ _ 

F-16  Hours:  _ _ _ 

Hours  Flying  with  Radar  Altimeter:  _ _ 

Total  Flight  Time  (Include  Student  Hours):  _ 
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PILOT  QUESTIONNAIRE 


I.  Do  you  feel  the  "pull-up"  warning  alarms  were  initiated  for  the  most  part 

1.  Too  early 

2.  Slightly  early 

3.  Just  Right 

4.  Slightly  late 

5.  Too  late 


2.  Did  the  warning  system  get  your  attention? 

1.  Yes,  all  the  time 

2.  Yes,  most  of  the  time 

3.  Occasionally 

4.  Seldom 

5.  Never 


3.  How  often  did  you  get  nuisance  warnings? 

1.  All  the  time 

2.  Most  of  the  time 

3.  Occasionally 

4.  Seldom 

5.  Never 


4.  Were  there  any  instances,  in  your  opinion,  where  the  system  provided 
no  warning  when  it  should  have?  Describe  the  circumstances. 


5.  What  do  you  feel  is  the  usefulness  of  a  GCAS  system? 

1.  Extremely  useful 

2.  Useful 

3.  Moderate  usefulness 

4.  Useless 

5.  Extremely  useless 
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6.  How  often,  if  ever,  would  you  tend  to  over  rely  on  a  GCAS  warning 
system  (i.e. ,  using  the  system  as  a  pull-up  cue  during  a  boobing 
mission)? 

1.  All  the  time 

2.  Frequently 

3.  Occasionally 

4.  Seldom 

5.  Never 


7.  How  would  you  rate  the  adequacy  and  suitability  of  the  GCAS  warning? 

1.  Excellent,  optimum  warning  system 

2 .  Very  good 

3.  Good 

4.  Satisfactory,  but  improvements  could  be  made 

5.  Satisfactory,  but  improvements  are  essential 

6 .  Poor 

7.  Very  Poor 


8.  Were  there  any  design  considerations  omitted  that  you  consider  essential 
to  GCAS? 


9.  General  comments  on  GCAS 


10.  Comment  on  the  adequacy  of  this  simulation  for  rating  a  Ground 
Collision  avoidance  system  algorithm.  _ 


QUESTIONNAIRE  DATA 


Q.  (1.)  Were  there  any  instance!,  in  your  opinion,  where  the  system 

provided  no  warning  when  it  should  have?  Describe  the 
circumstances. 


Pilot  No.  1:  Yes,  on  the  low  level  mission  (No.  11),  I  skimmed  the  ground 
with  no  warning  at  the  very  beginning. 

Pilot  No.  2:  No. 

Pilot  No.  3:  No. 

Pilot  No.  4:  No. 

Pilot  No.  5:  No. 

Pilot  No.  6:  No. 

Pilot  No.  7:  No. 

Pilot  No.  8:  No. 

Pilot  No.  9:  When  flying  into  rising  terrain  (but  it's  not  meant  to 

provide  that  protection).  It  also  seemed  slow  to  react 
to  a  level  bunt  into  flat  terrain. 


Q.  (2.).  Were  there  any  design  considerations  omitted  that  you 
consider  essential  to  GCAS? 

Pilot  No.  1:  No. 

Pilot  No.  2:  Only  one,  maybe  looking  out  in  front  of  the  jet  some  would 

help,  if  possible. 

Pilot  No.  3:  None. 

Pilot  No.  4:  One  improvement  discussed  would  provide  an  optical  vernier 

representation  of  an  approaching  pull-up  call 
(i.e.,  how  close  am  I  to  getting  a  pull-up  call). 

Pilot  No.  5:  Pilot  warning  as  to  need  to  pull-up  prior  to  it  being 

critical  to  pull. 

Pilot  No.  6:  More  timely  voice. 

Pilot  No.  7:  Wider  looking  radar  Altimeter.  Predictive  curvature  at 

ranges  for  ridge  crossings  (i.e.,  when  inverted  ridge 
crossing  needs  to  predict  curve). 

Pilot  No.  8:  Yes,  there  should  be  a  method,  such  as  f award  radar 

altimeter,  to  include  terrain  in  front  in  the  algorithm. 
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This  would  eliminate  most,  if  not  all,  nuisance  warnings. 

Pilot  No.  9:  A  clear  signal  distraction  when  GCAS  comes  on  in  the  AUTO 
mode.  Brief  glitches  leave  you  wondering  what  happened. 

Q.  (3.)  General  consents  on  GCAS. 

Pilot  No.  1:  System  works  well  as  long  as  no  trim  inputs  caused.  Need  to 

correct  ridge  crossings  so  pilot  doesn't  get  too  many 
nuisance  warnings  during  ridge  crossings. 

Pilot  No.  2:  Seemed  to  work  quite  well.  Only  problem  was  with  nuisance 

warnings  after  ridge  crossings. 

Pilot  No.  3:  A  good  system,  needs  further  testing/changes  for  false 

warnings  after  ridge  crossings.  Also,  false  warnings  when 
flying  alongside  a  ridge.  It  appeared  that  the  radar 
altimeter  was  looking  sideways. 

Pilot  No.  4:  Good  system.  Due  to  less  than  optimum  VIS,  the  system 

appeared  to  have  good  potential.  If  the  pilot  can  turn  the 
system  on/off,  it  would  be  great  in  some  circumstances. 

That  is,  missions  flown  into  the  sun  in  the  desert  where 
hills  blend  into  the  shadows,  etc..  Ridge  crossings  seem 
to  trip  the  system  too  much. 

Pilot  No.  5:  Manual  system  worked  pretty  well  as  a  heads  up  call  to  a 
distracted  pilot;  even  though  the  system  had  problems  on 
the  downhill  side  of  a  r'dge  crossing  (false  warning). 

Pilot  No.  6:  Needs  work  on  ridge  crossings. 

Pilot  No.  7:  Good  concept.  Eliminate  false  warnings  and  will  be  helpful. 

Pilot  No.  8:  GCAS  is  essential.  I  think  this  system  is  a  good  start.  I 

think  you  could  put  this  present  algorithm  in  the  F-16  today 
and  it  would  be  adequate,  although  pilots  would  be  apt  to 
complain  about  the  nuisance  calls.  I  theorize  pilots  might 
actually  fly  their  aircraft  differently  so  as  to  minimize 
nuisance  warnings.  The  only  way  to  eliminate  these  false 
warnings  is  to  have  a  f oward-looking  source  as  well  as  the 
standard  radar  altimeter.  One  other  conment:  the  two 
second  reaction  time  may  be  too  long  in  the  algorithm.  It 
was  hard  to  simulate  this  since  I  was  expecting  many  of  the 
warnings  but  in  the  aircraft,  T.  think  a  pilot  might  react 
slightly  quicker  than  two  seconds  to  an  unexpected  warning 
(pull-up  now,  ask  questions  later).  Over  all,  it's  a  good 
system! 

Pilot  No.  9:  Needs  to  be  given  without  delay.  The  false  warnings  on 

ridges  were  less  frequent  than  I  expected  so  the  warnings 
that  did  come  on,  occasionally  took  me  by  surprise. 
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Q.  (4.)  Comment  on  the  adequacy  of  thia  simulation  for  rating  a 

Ground  Collision  Avoidance  System  algorithm. 

Pilot  No.  1:  Will  cut  down  on  CFIT,  but  I  don't  think  the  system  will 
completely  eliminate  the  ground  collision  problem. 

Pilot  No.  2:  Very  good.  Naturally,  since  you  can't  have  perfect  visual 

simulation  you  can't  duplicate  conditions  perfectly,  but  it 
seemed  to  work  fairly  well. 

Pilot  No.  3:  None. 

Pilot  No.  4:  Good.  The  lack  of  periferal  vision  makes  turning  at  low 

altitude  and  performing  additional  tasks  tough. 

Pilot  No.  5:  Very  good.  The  simulator  flies  much  like  the  jet. 

Pilot  No.  6:  Good. 

Pilot  No.  7:  Good. 

Pilot  No.  8:  Fairly  poor.  Biggest  problems:  visual  perceptions  of 

depth,  lateral  position,  and  poor  lighting.  It  is  quite 
possible  that  one  may  have  almost  no  nuisance  warnings  if 
the  aircraft  waa  flown  (over). 

Pilot  No.  9:  The  profile  was  great. 


Q.  (5.)  How  does  the  "slope”  warning  compare  to  the  pull-up  call? 

Pilot  No.  1:  Good  for  level  terrain,  but  useless  with  really  steep 

ridges. 

Pilot  No.  2:  I  like  it  well  enough,  but  at  low  altitude  it  could  give  you 

a  warm  feeling  falsely  when  you  in  fact  should  be  getting  a 
"Pull-up."  Could  be  very  useful  for  insidiously  rising 
terrain. 

Pilot  No.  3:  Pull-up  is  all  that  is  necessary. 

Pilot  No.  4:  I  like  "UPSLOPE."  It  is  less  harsh,  a  remark  rather  than  a 

command. 

Pilot  No.  5:  The  slope  call  seemed  kind  of  useless  in  that  it  came  on 

about  one  second  before  the  pull-up  command.  The  ups lope 
should  either  occur  much  earlier  or  not  at  all.  (That  is  to 
say,  only  have  the  pull-up  command.) 

Pilot  No.  6:  Use  pull  up  all  the  time. 

Pilot  No.  7:  No  value.  Either  call  brings  your  attention  inside  and 

evaluate  the  situationd.e. ,  you  are  not  going  to  start  and 
pull  unless  you  know  your  attitude). 

Pilot  No.  8:  Not  necessary.  It  defines  too  narrow  a  range  of  conditions, 
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and  almost  overlaps  the  already  adequate  pull-up  call. 
Besides,  pilot  reaction  would  be  the  seme,  so  why  use  two 
warnings  ? 

Pilot  Mo.  9:  I  did  not  see  much  utility  in  the  slope.  The  actions  were 
the  same,  so  I  would  stay  with  one. 
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MISSION  BRIEFINGS 


MISSION  11  -  LOW  LEVEL  NAVIGATION 


OBJECTIVE:  Evaluate  GCAS  algorithm  for  nuisance  warnings /pull -ups  during 
aggressive  low-level  maneuvering. 


SCENARIO:  This  is  a  low-level  nav  mission  planned  at  300  to  500  feet, 
480  knots.  Fly  the  nav  profile  as  closely  to  the 
planned  line  as  possible.  Threats  are  active  above 
800  AGL.  Fly  as  aggressively  as  you  can  to  stay  below 
500  feet.  You  have  programmed  TOS's  for  each  steerpoint. 


START:  Altitude:  2200  MSL 

Groundspeed:  480  Knots 
Heading:  210 


INSTRUCTIONS: 

1.  Make  your  TOS  (Groundspeed  cue  will  help). 
2*  Stay  below  800'  or  get  zapped! 

3.  Evaluate  each  "pull-up"  call  as: 

A.  Valid 

B.  Invalid,  expected 

C.  Invalid,  unexpected 
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MISSION  12  -  LOW  SPEED  MANEUVERING,  IMC 


OBJECTIVE:  Evaluate  GCAS  low  speed  effectiveness  against  varied  terrain. 


SCENARIO:  You  have  just  completed  an  enroute  descent  to  Buzsard 

Creek  AFB.  Since  you  lost  the  approach  plate,  you  have 
asked  for  radar  vectors  to  a  PAR  final.  The  controller  has 
assigned  you  an  altitude  of  1800  feet  MSL.  That  sounds  pretty 
low,  but  you  think  you  remember  that  the  MSA  was  1400  feet. 

You  are  in  IMC.  Your  radar  altimeter  is  busted. 

START:  Altitude:  1800  feet  MSL 

Airspeed:  250  KCAS 
Heading:  268  degrees 


INSTRUCTIONS: 

1.  Maintain  250  KCAS  and  follow  the  steerpoint  cues. 

2.  If  you  get  a  pull-up  alert,  pull  as  hard  as  you  can — use 
power  as  you  feel  the  situation  dictates. 

3.  Climb  to  3000  MSL  for  15  seconds,  then  descend  expeditiously 
to  1800  feet  again. 


MISSION  13  -  HARD  TURNS  AT  LOW  ALTITUDE 


OBJECTIVE.  Evaluate  GCAS  effectiveness  during  hard  defensive  maneuvering 
at  low  altitude. 


SCENARIO;  You  are  practicing  hard  defensive  turns  at  low  altitude. 

You  will  fly  a  figure  8  pattern  between  Millersburg  and 
Elisabethville.  To  keep  you  honest,  a  "SAM"  threat  will  sound 
each  time  you  go  above  800  AGL. 


START:  1000  feet  MSL  (500  AGL) 

Heading:  094 

Airspeed:  480 


INSTRUCTIONS: 

1.  At  each  steerpoint,  make  a  hard  (5~G)  turn  back  toward 
the  next  steerpoint. 

2.  Always  turn  toward  the  north,  away  from  the  mountains. 

3.  Each  time  you  turn,  you  will  be  instructed  to  read  a 
series  of  numbers  and  letters  from  a  card  located  above 
your  head  to  simulate  checking  6.  Try  to  maintain  your 
G-load  and  altitude  while  you  read  the  data. 

4.  If  you  get  a  warning,  recover  and  continue. 
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MISSION  14  -  RANGE  MISSION 


OBJECTIVE:  Evaluate  GCAS  ef fectivenesa  during  a  complex  range  mission 
with  aide  taskiug. 


SCENARIO:  This  is  your  first  trip  to  the  Berrysburg  Boobing  Range. 

You  are  trying  out  a  number  of  alternate  run-ins  for 
straight  ahead  pop-up  deliveries.  Because  of  a  peculiar 
local  airspace  restriction,  you  oust  remain  at  or  below 
500  AGL  except  in  the  target  area. 


START: 


Altitude:  1300  feet  MSL 

Airspeed:  430  knots 
Heading:  149 

Weapx.s:  4  MR  82’ a.  stns  3  7, 

CCIP/S ingles  RP  -  1 


INSTRUCTIONS: 

1.  Make  4  straight  ahead  pop-up  attacks  on  the  Berrysburg 
target. 

2.  The  sequence  of  the  runs  is  on  your  map.  After  each  run, 
fly  to  the  bend  in  the  river  and  then  along  Hoof  lander 
Mountain. 

3.  If  you  are  in  doubt,  follow  the  steerpoints.  Console 
operator  will  help. 

4.  Make  tight  turns  (at  least  3  G'a). 

5.  Start  your  pull-up  when  you  are  20  seconds  from  target. 

-  pull  up  30  degrees. 

-  pull  down  at  4000  feet  MSL  (30  degrees). 

-  release  at  2500  feet  MSL. 

-  do  your  best  not  to  short  the  pass  —  even  if  you 
feel  a  bit  steep  (unless  you  get  a  pull-up  warning). 

6.  Stay  in  the  NAV  Master  Mode  until  you  are  on  run-in  heading, 
then  select  A/G. 

7.  You  will  be  asked  to  perform  additional  tasks  during  the 
mission. 

8.  If  you  get  a  pull-up  warning,  recover  and  continue  the 
mission. 
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MISSION  15  -  LOW  LEVEL  NAV/POP-UP  DELIVERIES 


OBJECTIVE:  Evaluate  the  GCAS  effectiveness  during  low  altitude  tactical 
navigation/weapons  delivery. 


SCENARIO:  This  is  a  coabined  nav /weapons  delivery  profile.  There  are 
three  targets  planned  for  straight  ahead  pop-up  deliveries. 

The  targets  are  fairly  easy  to  see:  a  bridge,  a  ship*  and 
an  airfield.  The  area  is  heavily  defended.  If  you  climb 
above  800  feet  AGL,  you  will  get  a  "SAM"  warning. 

START:  Altitude:  2100  feet  MSL  (500  feet  AGL) 

Airspeed:  480  knots 
Heading:  054 

Weapons:  6  Mk  82 'a/Pairs /RP  1/CCIP 

INSTRUCTIONS : 

1.  Fly  480  knots  G's  or  the  speed  required  to  make  your 
TOT.  Turns  are  60  degrees  bank.  Stay  at  300  AGL 
except  during  weapons  delivery.  Maintain  480  feet 
during  target  run  on. 

2.  Weapons  Delivery: 

TGT  Pull-Up  Pull-Down  Release 

1  (Bridge)  30  degrees  at  20  sec  4100  feet  2600  feet 

2  (Ship)  30  degrees  at  20  sec  4000  feet  2500  feet 

3  (Airport)  30  degrees  at  20  sec  4000  feet  2500  feet 

3.  If  you  get  a  pull-up  warning,  recover  and  continue  the 

mission  at  the  planned  altitude. 

4.  If  you  get  a  warning  during  NAV  leg,  evaluate  it  as 
either: 

A.  Valid 

B.  Invalid,  expected 

C.  Invalid,  unexpected 


MISSION  16  -  IMC  MANEUVERING 


OBJECTIVE:  Evaluate  the  ayatea'a  ef fectiveneas  in  recovering  from 
situations  where  spatial  disorientation  is  a  factor. 


SCENARIO:  You  will  perform  a  number  of  aerobatic  maneuvers  in  IMC. 


NOTE:  A  non-standard  HUD  Bank  display  will  be  used  to  help  your 

attitude  awareness. 


INSTRUCTIONS: 

1.  Stay  within  10  nai  of  steerpoint  "l".  Try  to  work  back  and 


forth  between  STPT  1  and  2. 

2.  Perform  the  following  maneuvers: 


A/S 

ALT  (feet) 

G's 

POWER 

Split  S 

300 

8000 

G’s  and 
maintain 

power  as  required  to 
300  knots. 

Loop 

500 

3000 

5 

MIL 

Cuban  8 

500 

3000 

5 

MIL 

S liceback 

350 

8000 

3-5 

MIL 

Double 

Split  S 

300 

14000 

G's  and 
maintain 

power  as  required  to 
300  knots. 

3.  If  you  get  a  pull-up  warning,  complete  the  recovery  to 
a  safe  altitude  above  5CG0  feet  and  set  up  for  the  next 
maneuver. 


77 


REFERENCES 


1.  Linquiat,  E.F.,  Design  and  Analytic  of  Experiaents  in  Psychology  and  in  Education. 
Boston:  Houghton  Miff iin  Co, ,  1953. 

2.  ASD/AG,  Draft  Statement  of  Work.  14  March  1986. 

3.  Cubic  Corporation,  Feasibility  Demonstration  of  a  Crc  ind  Collision  Avoidance  System, 
Tachnical  Report  414-1A,  1  September  1985. 


78 


